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1995 Annua! Report 
of the 

ERIC Clearinghouse on Assessment and Fvaluatic;. 

Executive Summary 

The ERIC Clearinghouse on Assessment and Eva uation at the Catholic University of 
America is one of 16 subject-oriented clearinghouses operated by the United States 
Department of Education, Office of Educational Research and Improvement. Since its 
inception in 1966, the Educational Resources Information Center (ERIC) has become one cf 
the major bibliographic databases in the world. ERIC has acquired, reviewed, and processed 
more than one-half million citations that policy makers, program planners, rese^chers, and 
other users can readily identify and obtain. 

I>; its early years, ERIC was primarily an archive of the education literature. Its prime 
activity was the development of its databases - Resources in Education and Current Index to 
Journals in Education. While this continues to be the cornerstone for all clearinghouses, 

ERIC has evolved into a much more powerful and useful resource. ERIC/AE and the other 
clearinghouses are heavily involved in providing a strong value-added service. We prepare 
syntheses of research on topics within our subject area, provide easy access to quality 
material, and serve as a central clearinghouse of scope-related activities. In the vernacular of 
information science, ERIC and ERIC/AE are information providers. 

The explosive growth of the Internet has been accompanied by an increased demand for 
information. ERIC as a system is now widely recognized as the central source for 
information in the education and social science fields. We have worked to promote ERIC/AE 
as the central source for assessment and evaluation information. We have created numerous 
partnerships for electronic dissemination of information, created active gopher and web 
pages, established one listserv while participating in others, created pathfinders to help 
patrons locate information, prepared articles on Internet resources, and given several well 
attended workshops. 

""his annual report details our 1995 activities. It discusses how we acquire documents, 
from whom we acquire them, the processes of selection and input for the ERIC databases, 
user services, publications, management activities, and partnerships. 

In 1995, we emphasized network building. We formed partnerships with non-profit 
organizations, commercial firms, book publishers, professional associations, university 
professors, and individuals. The common goal of these partnerships is to aid in the 
distribution of information and materials, and to teach about ERIC, ERIC/AE, and the 
Internet. These partnerships have made us into a central clearinghouse for information as 
well as documents. In 1995, we received more than 236,000 requests for information. This is 
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100-fold over the target established when we were awarded the clearinghouse operation just 
two and one-half years ago. In 1995 we distributed over 85,000 ERIC digests and more than 

145.000 other full text resources. 

A key partner in our activities has been the Adjunct Test Collection Clearinghouse at the 
Educational Testing Service. The Adjunct Clearinghouse provides ERIC resumes of test 
descriptions. ' r hese descriptions form the basis of the ERIC/ETS/Buros/Pro-Ed Test Locator 
Service. In 1995, the Test Locator had over 34,000 users, who conducted approximately 

78.000 searches and examined over 93,000 test descriptions. 

We believe this increased visibility and service to the broad education community has 
aided us in our acquisition efforts. Conference coordinators cooperate in acquiring materials 
for the ERIC System by printing the ERIC Reproduction Release Form in conference 
programs. Several organizations actually acquire documents for us. In 1995 we considered 
2,157 documents for inclusion in Resources in Education. Many of these documents were 
transferred to other clearinghouses. 

ERIC/AE was also active in ERIC database building. We submitted 812 documents for 
inclusion in Resources in Education and 880 for the Current Index to Journals in Education. 
These numbers are significantly higher than our targets of 700 each. 

We produced 11 Digests in 1995: Basic Item Analysis for Multiple-choice Tests ; Writing 
Multiple-Choice Test Items; More Multiple-choice Item Writing Do’s And Don’ts; The 
Program Evaluation Standards; Grading Students', Making the A: How To Study for Tests ; 
Cost of a National Examination; Student Ratings Offer Useful Input to Teacher Evaluations', 
Item Bias Review; Assessment & Evaluation on the Internet ; and Goals 2000 and World-Class 
Standards on the Internet. Several address practical issues confronted by teachers and 
assessment professionals; others identify resources and policy issues. 

Two initiatives undertaken in 1995 focus on user outreach. ERIC/AE forged partnerships 
with 12 commercial and non-commercial publishers to create a clearinghouse of book titles, 
Testing in America. The catalog presents descriptions of 40 carefully selected titles in the 
field of assessment which patrons may order directly from ERIC/AE. In 1995 we also 
initiated our Education Partners Program for Professors of Educational Research, modeled 
after the highly successful partners program of the ERIC Clearinghouse on Counseling and 
Guidance. The program is designed to promote the use of the ERIC system through the 
carefully defined responsibilities of ERIC/AE and Research Professor Partners. 

Finally, ERIC/AE has been quite active within the ERIC System during 1995. We 
spearheaded an AskERIC users questionnaire to obtain baseline and evaluative information 
about the system’s e-mail reference service, and made numerous presentations about ERIC to 
a wide range of audiences. 
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1995 Annual Report 
of the 

ERIC Clearinghouse on Assessment and Evaluation 

Introduction 

The 1995 Annual Report of the ERIC Clearinghouse on Assessment and Evaluation 
comprises four major sections. The first. Document Processing, focuses on ERIC/AE 
activities in collecting, selecting, and processing materials tor inclusion in the ERIC 
publications Resources in Education and Current Index in Education. 

User Services highlights ERIC/AE’s activities in direct user support such as development 
of reference materials and search guides, responses to patron requests for information, 
workshops, and conferences. This section also includes descriptions of our efforts to educate 
patrons in the appropriate and correct usage of reference services, particularly those 
generated by the rapidly evolving electronic environment; this includes our Internet activities 
on bstservs, gopher sites, and World Wide Web pages. We also describe our connect' ons to 
professional organizations and provide information on our formal partnerships. 

The Publications section describes the publication process and our endeavors ii this area. 
This section also presents descriptions of ERIC Digests generated through ERIC/AE in 1995. 
We describe Testing in America, the catalog of current literature in testing and assessment. 
The catalog was developed to provide patrons with a single source for ordering important 
publications in the field. 

ERIC System Maintenance and Improvement describes our participation in ERIC 
administrative activities. These encompass review of terms for inclusion in the ERIC 
controlled vocabulary, and contributions to development of an AskERIC users survey and 
the ERIC Brag Sheet, a copy of which is attached. 

Appendix A contains names of organizations with which ERIC/AE arranged to solicit 
papers at meetings or conferences, or with which we have Standing Acquisition 
Arrangements. Appendix B is a listing of the journals routinely reviewed for inclusion in the 

Current Index to Journals in Education. 

I. Document Processing 

A. Resources in Education (RIE) 

Resources in Education announces recent nonjoumal reports in education. The 
announcements, in the form of content abstracts and indexes, provide information for 
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accessing articles by Subject, Personal Author, Institution, Publication Type, and 
Clearinghouse/ED number. ERIC releases documents cited in RIE in both microfiche and 
paper or microfiche only. The determination of release format is made based on source, 
copyright, or ownership requirements, or for legibility and reproducibility reasons. 

ERIC/AE is responsible for submitting 700 resumes per year to RIE. In 1995, we 
acquired 2,157 documents, 865 of which were within our scope. Of these, 812 were accepted 
for processing. While the apparent rejection rate was 6 percent, the actual rejection rate was 
higher, as borderline documents rejected by ERIC/AE are routinely transferred to other 
clearinghouses for second consideration. 

All documents that ERIC/AE acquire for consideration for inclusion in RIE are evaluated 
using high standards of quality. These standards, described in Eissenberg (1989), include 
technical adequacy, contribution to knowledge and sigmficance, relevance, new applications 
of knowledge and innovative practices, effectiveness of presentation and thoroughness of 
reporting, responsiveness to current priorities, timeliness, authority of author, source, or 
sponsor, and comprehensiveness. 

We receive approximately 50% of our documents in April and May because conferences 
held in those months provide such a large proportion of acquisitions for RIE. However, we 
are asked to transmit comparable numbers of documents to the facility each week, and the 
nature of the contract dictates that we not greatly exceed the quantities of documents 
specified in the task order. As a result we usually have at least a one month backlog of 
documents ready to be shipped to the facility . 

Routine documents acquired during 1995 required an average of 19 days from time of 
document receipt to shipment to the abstractors; 14 days for abstracting; and 50 days from 
receipt of the citations to shipment to the facility. Processing of priority documents averaged 
10 days from time of receipt to time of shipment. We have processed priority documents in 
as few as 3 days. 

B. Selection and Input for Current Index to Journals in Education (CIJE) 

Current Index to Journals in Education is a guide to recent journal articles in more than 
800 major education-related joumals.During 1995 ERIC/AE processed 880 articles from 34 
professional journals, once again exceeding our goal of 700 resumes. 

Selection of journal articles to be processed is based on the article’s contribution of 
substantive new material to the body of educational information, its relationship to the le 
of education, and its comprehensiveness. Book reviews, editorials, letters to the editor and 
the like are not normally selected. 

The average turnaround time between journal articles being received and their resume 
being transmitted to the ERIC Facility was 33 days in 1995. As with Resources in Education , 
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we pace our transmissions to assure an even flow of documents. We typically have a two or 
three week backlog of prepared resumes. 

II. User Services 

A. Web ar ' Gopher Activities and Statistics 

We view the Internet as our opportunity to become a central clearinghouse of information 
about assessment and evaluation. Toward that end, we maintain a web page pointing to all 
known assessment and evaluation resources, volunteer to post material, maintain a listserv, 
and post a wide range of information within our subject area. As illustrated by the attached 
table, we have been quite successful in our efforts. 



ERIC/AE Gopher Transactions 
January 1 - December 31, 1995 


Activity 


count 


Logins 


141,653 


Retrieved directories 


226,802 


Retrieved files 


105,173 


Searches 


140,196 


Retrieved from search 


219,409 


Retrieved binary files 


594 


Total Transactions 


833,727 



A login is counted as a new user if more than one-half hour has elapsed since the user 
last entered the site. This provides a more realistic index than root connect count because 1) 
many users bypass the root to enter our gopher space, and 2) root connects are artificially 
high as the same user usually accesses the entry menu several times while exploring the site. 

In the above chart, retrieved files are text files — mostly digests (25,035), newsletter 
columns, and information sheets. Searches are primarily searches of the ERIC database, Test 
Locator, and Digest database. Retrieved from search are mostly digests (31,851) and test 
descriptions (93,293). Retrieved binary files are downloaded software pertaining to 
assessment and evaluation and downloaded papers from our Full Text experiment as 
discussed in Professional Associations, AERA. 
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Our statistics for web usage, presented in the next table, are estimates based on a recently 
installed counter on our most popular web pages. The first and last listed web pages (home 
page and Test Locator) are primarily front-ends to our gopher. The other two items, 
however, are terminal points directing the user to other Internet sites; counts from them can 
be added to the gopher logins to provide a good estimate of the number of ERIC/AE Int ernet 
users. 



Web Page Users 

(monthly and estimated annualized counts) 


Activity 


count 

(monthly) 


count 
(est annual) 


ERIC/AE home page 


4,230 


44,838 


Search ERIC 


6,480 


68,688 


Internet Resources 


2,160 


22,896 


Test Locator 


1,500 


15,900 



To put these web page counts in perspective, the Medical Center Home page gets 6,000 
users per month; X-files home page gets 4,500; Toyota Camry gets 4,300; Ham Radio 
Online Magazine gets 3,000; and the American Educational Research Association home page 
gets 2,000 users per month. In February 1996, the Search ERIC page was up to over 9,000 
users per month. 

Combining the gopher login count and relevant estimated annualized web statistics, the 
estimated total number of ERIC/AE Internet users was 233,237 in 1995. We are most 
excited by the counts of retrieved text files: the full text materials posted by ERIC/AE 
(digests, essays, newsletter articles, standards) were accessed over 231,189 times. We feel 
that this unprecedented amount of information utilization provides us with new 
responsibilities and opportunities. Accordingly we will continue to devote significant 
resources toward our Internet activities. 

B. Test Locator 

The Test Locator is a joint project of the ERIC Clearinghouse on Assessment and 
Evaluation, the Library and Reference Services Division of the Educational Testing Service, 
the Buros Institute of Mental Measurements at the University of Nebraska in Lincoln, and 
Pro-Ed test publishers. It is composed of the ETS Test Collection, Buros Test Review 
Locator, Pro-Ed Test Review Locator, and the Pro-Ed Test Publisher Locator. While each 
component of the Test Locator is a major service in and of itself, putting them together 
provides the end-user with more than the sum of the parts. In 1995, the Test Locator had 
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over 34,000 users. They conducted approximately 78,000 searches and examined over 

93.000 test descriptions. 

The Educational Testing Service (ETS) Test Collection database contains records on over 

10.000 tests and research instruments. These records describe the instruments and provide 
availability information. The Test Collection encompasses virtually all fields including social 
skills inventories for adults to education-related achievement and aptitude tests. The test 
descriptions are indexed with ERIC Thesaurus terms for subject accessibility. As an adjunct 
ERIC Clearinghouse, the ETS Library and Reference Services Division prepares the 
descriptions. ERIC/AE maintains the database, hosts the Search System, and puts the 
education related test descriptions into the ERIC database. 

Users may turn to the Buros Test Review Locator for direction to test reviews and 
descriptions in Buros Institute for Mental Measurements publications. Reviews are not 
online; references are given for die print editions of Mental Measurements Yearbook or Tests 
in Print. The cornerstone of the Buros Institute’s publishing activities, the Mental 
Measurements Yearbook ( MMY ), is a comprehensive set of consumer-oriented test reviews. It 
provides descriptive and evaluative information, and an extensive list of references to 
pertinent literature. Tests in Print (TIP) is as a comprehensive bibliography to all known 
commercially available, English-language tests currently in print. It contains pointers to 
reviews in MMY and a reference list of professional literature providing useful information 
about a specific test. 

The PRO-ED Test Review Database is an online index to Pro-Ed Test Critiques. Test 
Critiques offers the reader crucial information on each individual te. such as practical 
applications and uses, settings in which the test is used, appropriate as well as inappropriate 
subjects, and guidelines for administration, scoring, and interpretation. Entries cite normative 
data, validity and reliability information, and report what other experts ha\ . said about the 
test. The Test Publisher Locator, originally compiled by Pro-Ed, contains the last known 
addresses of 470 major test publishers. 

C. K12ASSESS-L 

K12ASSESS-L, an electronic forum for school based personnel, researchers, and others 
interested in issues pertaining to the assessment of students in grades K.-12, was created by 
the ERIC Clearinghouse on Assessment and Evaluation in the Fall of 1995. The goal of 
K12ASSESS-L is to provide educators with a fast, convenient, and topical electronic 
discussion forum focusing on issues related to educational assessment in grades K-12. 
K12ASSESS-L is rapidly becoming the place for local assessment personnel to share and 
obtain resources, ideas, and support. There are currently more than 600 subscribers to 
K12ASSESS-L and the listserv averages about 5 posts per day. 

In establishing the listserv, we followed the lead of the AERA listservs and specifically 
stated that anything written to K12ASSESS-L would be in the public domain and available 
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for unlimited reproduction. This permits us to create summaries and Thread Gems (excerpted 
discussions on a given topic) which can be disseminated in print and electronic media. We 
have taken the unusual step of providing a home page for the iistserv (http://www.cua.edu/ 
www/eric_ae/kl2assess/). At the home page, users can subscribe, unsubscribe, review the 
welcome message, read about the Iistserv, examine the archive, and read Thread Gems. 

D. User Services Product Development 

A traditional tool of the library science trade is the pathfinder, an annotated bibiiography 
which steers users toward resources appropriate for a specific discipline. ERIC/AE has 
adapted the pathfinder to contemporary resources by creating one which identifies and 
describes electronically-available databases of assessment and evaluation related information. 
That pathfinder was published in Educational Measurement: Issues and Practice and is used 
on-line by 150 patrons per day. At the request of the ERIC Clearinghouse on Counseling and 
Student Services, we have developed a similar pathfinder for their discipline. 

Another important ERIC/AE product is the set of user instructions for accessing and 
navigating the ERIC Test Locator — a database of test descriptions, test publishers, and 
citations to scholarly reviews. 

For the significant number of requests from patrons who want general information about 
ERIC and the functions of our clearinghouse, ERIC/AE created a flyer describing 
ERIC/AE’s role in the ERIC System, our scope of interests, the roles we perform for the 
collection and dissemination of information, and our World Wide Web and gopher site 
resources. The flyer also lists important ERIC phone numbers and Internet addresses. 

E. Patron Requests for Information 

Patrons contact ERIC/AE via telephone, correspondence, and electronic mail to obtain 
information within our scope. In 1995, we received 2,548 e-mail queries, 923 
correspondence inquiries, and 612 Telephone inquiries. 

Computer searches are the primary basis of responses to all patrons, but they are not the 
sole information source consulted in the formulation of responses. ERIC/AE maintains a 
library of print resources including statistical compilations and test reviews which are also 
consulted. Responses sent by postal delivery typically include an information packet 
consisting of a cover letter which includes directions for accessing ERIC documents, 
printouts of ERIC database citations and abstracts; ERIC Digests, as appropriate; an 
appropriate ERIC Review; information on ERIC Clearinghouses, other Department of 
Education-sponsored projects; and other resources (including print and electronic resources) 
which may be useful. 
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Frequently asked questions of ERIC/AE: 

ERJC/AE receives a wide range of questions on a wide range of topics. We routinely 
review the most frequently asked questions to ensure we have identified the best available 
resources on the topic. What follows are examples of the most frequently asked questions of 
ERIC/AE. 



• Do you know of any instruments to measure x characteristic of y 

population? (e.g., student attitudes towards mainstreaming) 

• What is the predictive validity of the SAT? Does it discriminate according 

to gender and to ethnicity? What have been the trends in SAT scores in 
the last 20 years? 

• How do U.S. elementary and secondary students compare with their foreign 

counterparts on standardized tests of achievement? 

• Please provide information about alternative assessments, especially 

portfolio assessment, and methods of implementation. How do we 
implement portfolio assessment at x grade level? 

• Please provide information about Howard Gardner’s theory of Multiple 

Intelligences. How is the theory of Multiple Intelligences applied in the 
classroom ? 

• What is the appropriate age for assessment for giftedness? What are/how 

do I locate appropriate measurement instruments for this purpose? 

• Please provide general and specific information about block scheduling. 

What are the advantages/disadvantages of block scheduling? What are 
examples of other schools schedules which are based upon block 

scheduling? 

• What are advantages/disadvantages/models/evaluations of year round 
schools? 
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Sample Patron Thank Yous to ERIC/AE 



Subj: RE: Test Anxiety Citations 

THANKS for the info!!!! I love tne immediate results this service provides. 

**** *************************************************************** 



Thank you for searching for info on rubrics for me; the materials you provided did give a nice 
overview. You mentioned that searches can be conducted via the Internet. I would like directions 
as to how to do this; I am a fairly new e-mail user and have access to both IBM and MAC formats 

(both with 14400 modems) if that makes any difference. 

************** ********************* ************************ ********* 



Your instructions were excellent. It took several tries to get into HOLLIS, but I finally succeeded. 
I did find the abstract, and I want to thank you for your help. 

*********************** ********* *************************** ********** 



Thanks for your help. This was my first experience with an on-line search... I’m REALLY 
impressed. Great service especially for those of us who live close to 100 miles from nearest ERIC 
library. Thanks again for all the CASAS-research related material. It fits in with the project I’m 
working on for the class, plus a few other projects. 
********************************************************************* 



Thanks for the excellent information on standardized testing for 1st graders. Yes, I would enjoy 
doing my own ERIC search on the Internet and would appreciate if you would send directions on 
"how to". Thanks for your help. 

********************************************************************** 



I am writing to thank you for the ERIC search you completed for me. It has been very helpful in 
helping us determine a tool for assessing children. The lack of information on the BASC has helped 
us decide to try several assessment tools before mailing a decision. Again, thank you for 
responding. 

********************************************************************** 



Thank you for the information you sent to me regarding portfolio assessment. It is very helpful, 
and I will let you know if I have any further questions. I have already received the directions for 

how to conduct an ERIC database search via the Internet, so I will do that next. Thank you again! 
********************************************************************* 



I cannot thank you enough for your most valuable information. Since so many people had no 
never heard of the Copyright Clearance Center, I was beginning to wonder if I just dreamt about 
it. Thanks again! 
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F. AskERIC User Services 

The Clearinghouse on Information and Technology at Syracuse University operates 
AskERIC, an Internet-based service. Patrons send e-mail questions about education or 
educational issues to AskERIC which are then referred to specialists at the clearinghouse 
covering the topic. Subject specialists respond with basic reference citations and other 
materials tailored to the question within 48 hours. At the end of 1995, AskERIC was 
handling 550 questions per week of which ERIC/AE responded to approximately 50 per 
week. 

G. ERIC Workshops 

ERIC/AE conducts several workshops a year for audiences which include conference 
attendees, university students, and clearinghouse visitors. While the majority of our 
presentations promote general awareness of ERIC and ERIC/AE resources, requests are 
increasing for workshops on ERIC and other education related Internet resources. 
Approximately 700 people attended our workshops. 

In presenting workshops, we use an ERIC Slide show, created using Microsoft Power 
Point; the All About ERIC handbook; and relevant ERIC Digests. In addition to the slide 
show, the ERIC/AE presentation includes hands-on searches of the ERIC Database. The 
presentation covers a general explanation of ERIC, how to search ERIC effectively, ERIC 
System components, ERIC/AE, and ERIC resources on the Internet. Thumbnail images of 
selected slides are attached. A set of speaker’s notes accompanies each slide in the show, 
allowing the presenter to redesign the show to meet the expectations of the audience. The 
slide show is available on 35mm slides, color overheads, and a self-contained IBM PC slide 
show program. 

H. Conference on Assessment for Guidance Personnel 

The ERIC Clearinghouse on Counseling and Guidance and ERIC/AE collaborated in 
developing a conference for K-12 student services specialists which was held in January 
1995. The goals of the conference were to provide information on new developments in 
assessment and testing, social and political issues in assessment, and interprofessional 
collaboration. More than 300 people attended the conference, which featured nearly 20 
speakers. ERIC/AE helped design the conference, recommend speakers, and obtain speakers. 
In addition we prepared a Digest, made a large group presentation, and coordinated an 
information exchange at the conference. 

I. Linkages with Professional Organizations 

We have extensive links to the American Educational Research Association and the 
Educational Testing Service. These organizations make significant contributions of documents 
to our clearinghouse and the entire ERIC System. 
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1 . American Educational Research Association 

The American Educational Research Association (AERA) is the most prestigious and 
prominent international professional organization with the primary goal of advancing 
educational research and its practical application. Its more than 20,000 members are 
educators; administrators; directors of research, testing or evaluation in federal, state and 
local agencies; counselors; evaluators; graduate students; and scientists. The broad range of 
disciplines represented includes statistics, anthropology, and political science. 

ERIC/AE assists the AERA with support and advice on telecommunications projects and 
provides a workshop on ERIC and Internet for the AERA home office staff. As a part of our 
document acquisition activities, we have a booth at the AERA annual meeting and solicit 
papers. As a result, the AERA refers patrons to ERIC/AE for help in locating authors of 
papers presented at the annual meeting. 

In 1995, the ERIC Clearinghouse on Assessment and Evaluation worked very closely 
with the AERA and the National Council on Measurement in Education in establishing an 
FTP site to provide improved access to the papers presented at the annual meetings. 

ERIC/AE created a searchable database containing short descriptor files and pointers to the 
full text of presentation papers uploaded by presenters, making them available for 
downloading over the Internet. 

A copy of the joint AERA-ERIC/AE announcement for the AERA project appears on the 
following page. A similar arrangement was made with the National Council on Measurement 
in Education. The project was much more labor intensive than anticipated as few papers and 
descriptions arrived in the specified format. We decided not to repeat the project in the near 
future. 
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Make your paper available on the Internet 

AERA and the ERIC Clearinghouse on Assessment and Evaluation are pleased to 
announce a joint Internet project designed to facilitate the exchange of papers and 
information from the annual meeting. In addition to submitting a paper copy of your paper 
to ERIC, please submit an electronic copy of your paper and a description file. ERIC/AE 
will post the papers and descriptions at their gopher site (GOPHER.CUA.EDU under 
Special Resources/ Essays, Bibliographies & Resources/ AERA Annual meeting). You will 
also be able to obtain the papers from the AERA gopher site (INFO.ASU.EDU under 
ASU Affiliated Organizations/ AERA / Annual Meeting/). 

You may FTP your description and paper to FTP.EDUC.CUA.EDU, e-mail them to 
ERIC_AE@CUA.EDU (uuencoded), or mail them on diskette to AERA Papers, ERIC 
Clearinghouse on Assessment and Evaluation, Catholic University of America, 

Washington, DC 20064. 

The description should be in 70 column ASCII text format. The filename should be the 
session number, followed by the 3 letter initials of the first author and ".TXT", e.g. 
0102GLM.TXT. The first line should contain the file name and abbreviated title of the 
paper, e.g. 0102GLM - The counterbalanced equating and scale design. The description 
should then contain full title, author names, a 150 word abstract, information about the 
format of the paper (Word, WordPerfect, etc.), and e-mail and postal addresses. 

The second file should contain the full paper. Any format (Word, WP, Ami Pro, PC, 

Mac) can be used as long as everything is contained in a single file. This means that 
figures need to be embedded or the paper combined into an archive file format, such as a 
ZIP . Papers that are not in a compressed format will be ZIPped by ERIC/AE. 



AERA/ERIC FTP site announcement 



2. Educational Testing Service 

The Educational Testing Service (ETS) is a leader in educational research and testing. It 
is the world’s largest private nonprofit educational assessment and measurement institution 
serving a wide variety of clients in education, government, business, and the professions. 
While known primarily for the Scholastic Assessment Test (SAT), ETS is also known today 
for significant contributions to measurement science, computer-based testing technology, new 
modes of assessment, and a deeper understanding of teaching and learning. 
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In addition to the Test Locator project, we have had extensive contact and interaction 
with several divisions of ETS. During 1995, ERIC/AE arranged to archive the entire 
collection of ETS Research Memorandum. We also provided information on the SAT 
recentering and became a distributor of ETS produced material on test taking skills. Since 
ETS now provides us with unlimited quantities of their publications on test taking, ERIC/AE 
has combined digests prepared on the topic with the ETS materials to create an information 
package for responding to patron inquiries. 

J. Partners 

ERIC/AE maintains three formal partners programs. These partnerships are with 
professors of research, electronic publication partners, and book publishers (book publishers 
are discussed in the Publications section). 



1 . Electronic Partners 



Our electronic partners routinely provide us with materials for the gopher and web sites, 
aid in acquisitions, and aid in disseminating information about the clearinghouse. For 
example. Industry' Canada’s Community Access Initiative maintains a listserv dedicated to 
announcing and reviewing new useful websites for K-12 educators. We maintain a 
gopher/web page archiving their reviews and are co-administrators of their listserv. 



In 1995, we worked with the following electronic partners: 



American Educational Research 
Association 

Association for Institutional Research 
Buros Institute for Mental Measurements 
Educational Testing Service 
Reference Service Division 
FairTest 



Industry Canada’s C-Net 
Pacific Lutheran University 
PAR Systems User Group 
Portfolio Workbook 
Pro-Ed 

Rasch Measurement SIG 



2. Education Partners Program for Professors of Educational Research 



ERIC/AE’s Education Partners Program, introduced at the end of 1995, is designed as a 
collaborative effort between ERIC/AE and university staff. The program is designed to 
increase awareness of the ERIC system and to expand the use of systematic searches of the 
ERIC database and ERIC/AE resources and publications. It also seeks to develop the 
information research skills of faculty and students who enroll in measurement and/or research 
courses while promoting the sharing of education research resources. Our pioneering 
education partners are LeamWell Resources, Camino, California; Massachusetts Bay 
Community College; and Western Kentucky University. 
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As an affiliate of the ERIC system, the partner is entitled to benefits which include 
discounts or loans of some instructional materials. Partners may request master copies of 
ERIC/AE Digests for use as classroom or workshop handouts, and may request up to six 
customized searches of the ERIC database per semester to use as supplemental course 
bibliographies. As encouragement to become active contributors. Partners receive special 
"quick turn around" considerat'on of staff and student written materials tor inclusion in the 
ERIC database. 

In return, Partners agree to share responsiblity for ERIC education efforts, to promote the 
use of ERIC, and to be active contributors to the system. As their contribution to the 
assessment, evaluation and research literature base, partners agree to submit at least five high 
quality student or faculty written papers to be considered for input into the ERIC database. 

III. Publications 

A. Product development and review 

It is the publication series, more than any other activity, that makes the ERIC system 
more than a simple repository. ERIC publications get information into the hands of people 
who would not otherwise be able to obtain it. They also advance the state of educational 
practice by keeping people informed of the latest ideas and resources. Publications increase 
ERIC’s visibility as patrons use ERIC Digests in newsletters and as class handouts. Finally, 
the publications enhance the credibility of the clearinghouse, :.nd generate goodwill and 
improve acquisitions. 

To aid authors and to explain the Digest Development cycle, we have prepared a How to 
Write an ERIC/AE Digest. This booklet contains the OERI Digest guidelines and explains 
what Digests are, the required format, our editing policy, and our review policy. 

All ERIC/AE products go through several rounds of editing and review. We believe firmly 
that all documents can benefit from professional editing. Accordingly all ERIC/AE 
publications, including Digests, are edited to improve English usage and structure. In 
addition to routine proofreading, attention is given to grammar, presentation, structure, and 
transitions. The author reviews the edited version to assure subtle content changes have not 
been introduced and provides final approval. 
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B. Digests 

Digests are generated either by outside experts, or by in-house staff. Although Digests are 
an identifiable product line, it is possible to distinguish several, not necessarily mutually 
exclusive, types that address specific audiences and/or purposes. When developing Digests , 
we consider the types outlined by Central ERIC: Overview (introduction to a topic); Fact 
Sheet (current information of a factual nature related to a topic); Issue (defines and describes 
a controversial topic); Practice Application (provides specific, concrete examples of how 
practitioners can apply research results in practical settings); Research Findings (reports on 
the current status of research in an area); Synopsis of Synthesis Paper (summarizes an 
existing review and synthesis publication); and Digests. 

We distribute Digests free of charge to contributors of articles within our scope, test 
publisher’s representatives, and others asking to be placed on our Digest mailing list. We 
also routinely send Digests to conference coordinators for placement on exhibit tables. We 
often distribute Digests as a direct response to user inquiries. As shown in the following 
chart, we distributed 85,982 Digests during the year. 



ERIC/AE Digest Distribution 
January 1 - December 31, 1995 


Activity 


count 


On-line Digest database 


31,851 


Alternative Assessment on-line 


20,652 


Assessment and Evaluation on the Internet 


22,896 


Other posted AE Digests 


4,383 


Mail distribution 


6,200 


Total Transactions 


85,982 



Tides of ERIC/AE Digests Published in 1995 

1 . Basic Item Analysis for Multiple-choice Tests, Jerard Kehoe 

This Digest offers some suggestions for the improvement of multiple-choice tests using 
"item analysis" statistics. These statistics are typically provided by measurement services, 
where tests are machine-scored, as well as by testing software packages. The basic idea that 
we can capitalize on is that the statistical behavior of "bad" items is fundamentally different 
from that of "good" items. Of course, the items have to be administered to students in order 
to obtain the needed statistics. This fact underscores our point of view that tests can be 
improved by maintaining and developing a pool of "good" items from which future tests will 
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be drawn in part or in whole. This is particularly true for instructors who teach the same 
course more than once. 

2. Writing Multiple-Choice Test Items, Jerard Kehoe 

A notable concern of many teachers is that they frequently have the task of constructing 
tests but have relatively little training or information to rely on in this task. The objective of 
this Digest is to set out some conventional wisdom for the construction of multiple-choice 
tests, which are one of the most common forms of teacher-constructed tests. The comments 
which follow are applicable mainly to multiple-choice tests covering fairly broad topic areas. 

3. More Multiple-choice Item Writing Do ’s And Don ’ts, Robert B. Frary 

Writing Multiple-Choice Test Items (listed above) gives limited suggestions for 
item-writing due to its coverage of other aspects of test development. More Multiple-choice 
Item Writing Do 's And Don r ts presents a fairly comprehensive list of recommendations for 
writing multiple choice items. Some of these are backed up by psychometric research; i.e., it 
has been found that, generally, the resulting scores are more accurate indicators of each 
student’s knowledge when the recommendations are followed than when they are violated. 
Other recommendations result from logical deduction. 

4. The Program Evaluation Standards 

Sound evaluations of educational programs, projects, and materials in a variety of settings 
should have four basic attributes: utility, propriety, feasibility, and accuracy. The Program 
Evaluation Standards, established by sixteen professional associations, identify evaluation 
principles that when addressed should result in improved program evaluations containing the 
above four attributes. This digest is a summary of the Standards. 

5. Grading Students, Lawrence H. Cross 

Some instructors record letter grades for tests and assignments, and others record 
numerical values, often the percent correct on tests. Later, under either method, the grades 
are averaged, often employing a weighting process designed to make some grades count 
more heavily than others. Discussion of the merits of different approaches usually center 
around comparative advantages of averaging letter or numerical grades, or around some 
feature of the weighting process,. This Digest discusses several aspects of assigning grades. 
First, variability of test scores, an issue that underlies for both approaches, is discussed. The 
use of standardized scores is presented as a solution to the variability problem, ideas on 
assigning letter grades, and recommendations are then presented. 

6. Making the A: How To Study for Tests, Diane Loulou 

Tests are one method of measuring what you have learned in a course. Doing well on 
tests and earning good grades begin with good study habits. If your goal is to become a 
successful student, take the time to develop good study habits. This digest offers a plan to 
help you study for tests. It explains how to prepare for and take tests. Techniques for taking 
essay, multiple choice and other types of exams are reviewed. Although these techniques 
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may help you improve your test scores, other factors, such as class participation, independent 
projects and term papers also contribute toward grades. 

7. Cost of a National Examination 

Recent proposals from the federal executive branch and private groups have drawn 
unprecedented attention to the idea of a national examination for elementary and secondary 
students. The House Committee on Education and Labor asked GAO to look at school testing 
as it exists today, describe its nature, estimate its extent and cost, and assess how a new, 
national test might affect those factors. This digest provides a summary of Student Testing: 
Current Extent and Expenditures, with Cost Estimates for a National Examination. Report of 
the United States General Accounting Office, GAO/PEMD 93-8. 

8. Student Ratings Offer Useful Input to Teacher Evaluations, condensed from "Using 
Student Ratings in Teacher Evaluation, " by Michael Scriven 

Student ratings of instruction are widely used as a basis for personnel decisions and 
faculty development recommendations in post-secondary education today. This article 
addresses concerns about their validity and presents a case for the use of student ratings in 
teacher evaluation. 

9. hem Bias Review, Ronald Hambleton and Jane Rodgers 

When important decisions are made based on test scores, it is critical to avoid bias, 
which may unfairly influence examinees’ scores. Bias is the presence of some characteristic 
of an item that results in differential performance for individuals of the same ability but from 
different ethnic, sex, cultural, or religious groups. This dige? f introduces three issues to 
consider when evaluating items for bias — fairness, bias, and stereotyping. The issues are 
presented and sample review questions are posed. 

10. Assessment & Evaluation on the Internet, Liselle Drake and Lawrence Rudner 

With tens of thousands of information providers and millions of users, the Internet is an 
enormous and growing resource for measurement professionals, researchers, curriculum 
developers, evaluation personnel and others involved or interested in assessment issues. The 
task for busy professionals is to be able to rapidly identify Internet resources so they can be 
efficiently incorporated in their work. In this digest, we identify Internet resources of 
particular interest to the assessment community. Gopher sites and web sites are identified and 
discussed. 

11. Goals 2000 and World-Class Standards on the Internet 

Many organizations, schools districts, and state departments of education are posting 
useful materials on the Internet. These materials can prove to be a valuable resource for 
anyone interested in Goals 2000, World-Class standards, and school restructuring. The digest 
identifies key Internet offerings related to Goals 2000 and World Class standards. Pointers to 
each of these offerings can be found at the ERIC Clearinghouse on Assessment and 
Evaluation (ERIC/AE) gopher site (gopher.cua.edu, special resources, ERIC, Goals 2000). 
The broad target audience for these materials includes parents, teachers, school 
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administrators and others concerned with improving education. The information includes 
activities at the local, state and Federal levels. 



C. Catalog 

In order to provide patrons of our clearinghouse with current literature on topics of 
assessment and evaluation, we have cr ited a catalog of current book publications in our 
field. In October of 1995, ERIC7TM printed the first edition of Testing in America : A 
Collection of Titles from the ERIC Clearinghouse on Assessment and Evaluation. The catalog 
includes titles about general reference in testing standards, research and methodology, student 
evaluation, teacher evaluation, program evaluation, learning theory, educational reform, and 
the ERIC Database on CD-ROM. We believe the titles selected are outstanding in the 
addressed areas, and can help direct solutions to contemporary education problems. 



The catalog contains books by the following publishers: 



Addison-Wesley Publishing 
American Psychological Association 
Australian Council for Educational 
Research 

The Brookings Institute 

Buros Institute for Mental Measurements 

Corwin Press, Inc. 



FairTest 

Greenwood Publishing Group 
Oryx Press 

Sage Publications, Inc. 
Scholastic Inc. 

Teachers College Press 



All publishers included in the catalog agree to allow ERIC/AE to sell books to patrons at 
the publishers’ list prices. In turn, we buy the books from the publisher at a negotiated 
discount of 20% to 50%, returning the profits to the clearinghouse to pay for costs of 
production. Publishers’ distributors drop ship the orders directly to customers. Although all 
terms of the proposal are negotiable, only two aspects of the agreement are currently 
variable: some publishers prefer to send stock to us instead of drop shipping, and the 
discount varies. 



ERIC/AE markets Testing in America to superintendents in school districts, dioceses, 
regional centers, county superintendent offices, state departments of education, and public 
schools. Our next marketing project will be to post the catalog at the ERIC/AE home page 
on the World Wide Web; patrons will be able to browse the list of titles and submit orders to 
our book business manager via E-mail. 
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IV. ERIC System Maintenance and Improvement 

A. Vocabulary Review Procedure 

ERIC/AE participates in the development of the intemationally-renowned ERIC system 
controlled vocabulary. We discuss terms with the Vocabulary Review Group and expand the 
list of terms applicable to our clearinghouse scope. Because of the great interest in alternative 
forms of assessment and performance based assessment, ERIC/AE will be formally 
introducing several descriptors related to these concepts early in 1996. 

B. ERIC Task Force Activities 

ERIC/AE was active in several important system-wide initiatives during 1995. We 
participated in the Task Force which conducted the AskERIC Users Survey and an ad hoc 
group which developed the “ERIC Brag Sheet.” 

1. AskERIC Users Survey 

As kER IC is a system-wide service whereby patrons e-mail questions to ERIC and receive 
a response within 48 hours. Two ERIC/AE staff members led a Task Force which conducted 
a survey of AskERIC users. The goal was to obtain baseline data concerning who was using 
the service and their level of satisfaction. A questionnaire was piloted, revised and, in 
December 1994, distributed electronically to 400 AskERIC users. Results were tallied and 
released in February 1995. Usable responses were received from 101 respondents - 84 via 
e-mail and 17 by postal mail. The margin of error with this sample is between 3 and 10% at 
the 95% confidence level. 

Key Findings of the Study 

• Over two-thirds (68%) learned of AskERIC via a menu item at their local gopher, 
referral or workshop. 

• Almost all respondents (97%) would recommend AskERIC to someone else. 

• AskERIC users are primarily k-12 educators (68%), with half of the k-12 users being 
teachers and the rest evenly divided between administrators and librarians. 

• The most common reasons for using the AskERIC service were to improve classroom 
practice, professional interest, and help a colleague. 

• Only 5% of the respondents were using the service to help fulfill a school assignment. 

• Some 95% said they were satisfied with the speed of the response (which is 48 hours or 
less). Ninety-four percent felt the response was appropriate to their question. 

• Eighty-eight percent were satisfied with the quality of the response. 
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2. ERIC Brag Sheet 

Recognizing the need to develop a simple information sheet about ERIC, an ad hoc group 
spearheaded by ERIC/AE produced what has come to be referred to as the ERIC Brag Sheet. 
This information sheet, written in May 1995 and included as Appendix C, provides basic 
information about what ERIC does, who it serves, and where it is going. The Brag Sheet is 
now widely used as a short description of the ERIC system. 
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Appendix A: Conference- Wide Paper Solicitation and Standing 
Acquisition Arrangements 



Organizations Providing Papers from Annual Meetings and Conferences 



American Educational Research 
Association 

American Evaluation Association 
Canadian Evaluation Society 
Council of Chief State Schools Officers 
Eastern Educational Research Association 
Florida Educational Research Association 
Georgia Educational Research Association 
Hawaii Educational Research Association 
International Personnel Management 
Association 

Mid-South Educational Research 
Association 

Mid- Western Educational Research 
Association 

1995 Standing Acquisition Arrangements 

Akron Public Schools 
Albuquerque Public Schools 
American Council on Education 
American Educational Research 
Association 

American Evaluation Association 
Austin Independent School District 
Buffalo Public Schools 
Centet for Research on Evaluation, 
Standards and Student Testing 
Cleveland Public Schools 
Corpus Christi Independent School District 
Dade County Public Schools 
Dallas Independent School District 
Des Moines Public Schools 
East Baton Rouge Parish School Board 
Edmonton Public Schools 
Educational Testing Service 
Policy Information Center 
Research Department 



National Center for Education Statistics 

National Council on Measurement in 
Education 

New England Educational Research 
Organization 

Northeastern Educational Research 
Association 

Pennsylvania Educational Research 
Association 

Qualitative Interest Group 

Southwest Educational Research 
Association 

Texas Association for Institutional 
Research 

United Federation of Teachers 



Test Collection 

Florida Educational Research Association 

Fort Worth Independent School District 

Goteborg University 

Hartford Public Schools 

Home School Legal Defense Association 

International Association for the 

Evaluation of Educational Achievement 
International Personnel Management 
Association 

Jefferson County Public Schools 
Los Angeles Unified School District 
Mesa Public Schools 
Mid-South Educational Research 
Association 

Minneapolis Public Schools 
National Assessment Governing Board 
National Assessment of Educational 
Progress 

National Center for Fair and Open Testing 
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National Center for Research on 

Educational Accountability and Teacher 
Evaluation 

National Commission on Testing and 
Public Policy 

National Council on Measurement in 
Education 

National Research Center on Student 
Learning 

Nevada State Department of Education 
Newark Board of Education 
North Dakota University 
North York Board of Education 
Oakland Unified School District 



Palm Beach County School Board 
Philadelphia School District 
Rochester City School District 
Sacramento City Unified School District 
Saint Louis Public Schools 
San Antonio Independent School District 
Scarborough Board of Education 
South Dakota State Department of 
Education and Cultural Affairs 
Texas University 

Virginia Beach City Public Schools 
Wake County Public School System 
York Kegion Board of Education 
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Appendix B: Journals Reviewed for Inclusion in CUE 



Comprehensive Coverage: 

American Educational Research Journal 
Educational Research Quarterly 
Journal of Educational Measurement 
Journal of Educational Psychology 
Journal of Educational Statistics 
Review of Educational Research 
Studies in Educational Evaluation 
Applied Psychological Measurement 
Applied Measurement in Education 



Education: Comprehensive 

The Canadian Journal of Program 
Evaluation 

Cognitive Psychology 

Contemporary Educational Psychology 

Evaluation Practive 

Evaluation Review 

Intelligence 



Selective 

Assessment Update 
Canadian Journal of Education 
Evaluation and Program Planning 
Evaluation and the Health Professions 



Educational and Psychological 
Measurement 

Educational Evaluation and Policy 
Analysis 

Educational Measurement: Issues and 
Practice 

International Journal of Educational 
Research 

Journal of Personnel Evaluation in 
Education 



Learning & Instruction: The Journal of the 
European Association on Learning & 
Instruction 

New Directions for Program Evaluation 
Psychological Review 
Psychometrika 

Scandinavian Journal of Educational 
Research 



Journal of Experimental Education 
Assessment 

Journal of Youth and Adolescence 
Multivariate Behavioral Research 
Psychological Assessment 
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A Little About ERIC ( 1995 ) 

The Educational Resources Information Center (ERIC) is the nation’s information 
network connecting virtually ail educational information providers and educational 
information users. ERIC is a public service that uses technology to increase access to 
education research and practice to improve learning, teaching, and community-based 
educational decision-making. 

ERIC offers the substantive information people need. 

o The ERIC database includes summaries of nearly 900,000 of the best peer- 
reviewed documents and journal articles on education research and practice 
written since 1966. The ERIC database is the third most frequently used database 
in any field ( Computers in Libraries , February 1995). 

o Via the Internet, ERIC provides electronic access to a wide range of educational 
information and resources including full-text lesson plans and tests, thematic 
essays, reference material, and pointers to what others are doing. Over 30,000 
people log onto ERIC Internet sites, weekly. ERIC is frequently cited among the 
most useful places to visit {Internet World , January 1995; PC Computing , May 
1995). 

o ERIC currently produces and disseminates 1,500 briefing papers with over 150 
titles added annually. These syntheses provide balanced coverage of all the 
important education topics, including educational management, assessment, 
professional development, technology, and reform. 

o ERIC is a mediated education information system. Each year, the 16 subject- 
oriented ERIC Clearinghouses respond to over 50,000 email and telephone 
requests. 

ERIC has a tremendous audience. 

o Each year, more than 4,200,000 people use the ERIC database, the network of 
subject-oriented clearinghouses, and the various information services available 
through the Internet and commercial computer services. 

o Teachers in K-12 schools, universities, colleges, corporations, and public 
institutions consistently make use of ERIC information to help with decision- 
making and practical improvements in the education process. 
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ERIC brings efficiency and economy to the education community. 

o By offering a well-organized central repository for education information, ERIC 
streamlines educational decision-making. Local and state agencies naturally turn to 
ERIC rather than duplicating the costs that would be incurred if they had to 
research, identify, and collect material from multiple sources. 

o By identifying other organizations that have addressed similar issues, ERIC helps 
local and state education agencies learn from each other. 

ERIC leverages partnerships. 

o The federal investment forms the hub for many interconnected partnerships. These 
coordinated efforts with public and private partners enable ERIC to effectively and 
efficiently provide enhanced products and services. Heinmiller ( Cost and usage 
study of the ERIC System , 1981) estimated that each $1 of federal investment was 
leveraged to more than $20 in ERIC associated activity. 

o The ERIC system currently maintains formal partnerships with more than 400 
education organizations to aid in the development, announcement, and distribution 
of education information. 

o ERIC maintains no-cost-to-the-government corporate partnerships which provide 
print materials, document delivery as well as improved document acquisitions. 
These corporate partnerships include companies such as Computer Science 
Corporation, the Ford Foundation, the Educational Testing Service, and the 
American Bar Association. 

o Companies such as Apple ^ nputer and Sun Microsystems have made significant 
equipment donations. 

ERIC capitalizes on technology for continuous improvement. 

o ERIC has a history of technological innovation, first as the creator of the first 
commercial online database in the 1960s, then as an early adopter of CD-ROM 
technology in the 1980s. 

o ERIC now offers the award-winning AskERIC question-answering service and 
Virtual Library; the National Parent Information Network; and more than a dozen 
subject-oriented gopher /World Wide Web sites on the Internet. 

ERIC is highly customer-driven. 

o ERIC is beta testing an Internet site that will offer free database access and a 
powerful search engine to dozens of simultaneous users. 

o ERIC is moving toward delivering documents in full-text electronic format. 
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